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1. INTRODUCTION 

MPLS is an evolving connection-oriented switching technology based on labels instead of Internet 
Protocol (IP) address and is an undergoing study until today [1-7]. This technology creates virtual 
connections between two Label Edge Routers (LER) that are located at the edge of backbone network [8]. 
The backbone of the MPLS network consists of LER, Label Switch Router (LSR), and Label Switch Path 
(LSP). One role of LER is for Quality of Service (QoS) to ensure and distribute the labels to all the network 
components along unidirectional tunnel called the LSP [8-9]. It is also the first router to encapsulate and 
decapsulate by adding and removing labels onto the packets into the LSP and make the initial path selection. 

According to Shawl et al., the LSR comprises Label Distribution Protocol (LDP) that will be applied 
to the LSP as the signaling protocol for distributing labels throughout the MPLS network [9]. LSR has two 
main functions. Firstly, it is to ensure that the LSP is created. Secondly, it ensures that the data is transmitted 
according to the appropriate routing tables of labels that are created by LSR itself. When the LSR received a 
packet, it checks the routing table to find the next hop which is also an LSR. Then, it detaches the label and 
attaches a new one before it sends packet to the next hop [8, 10]. 

Currently, MPLS technology is meant for industrial purposes such as in teleproctection, Supervisory 
Control and Data Acquisition (SCADA), and protection relays to achieve low latency and high reliability 
[11-15]. Hence, the label switching mechanism can be configured freely only by the manufacturers and 
vendors. At the user side, MPLS routers can be configured based on limited options provided by the vendor 
to make sure the router is not vulnerable to hacking. 

Therefore, the freedom of reconfiguring of MPLS router is necessary for academic research 
purposes. Due to that reason, various of studies have been done by using simulation, commercial MPLS 
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router and embedded systems. This paper is aimed to compare and analyse the existing commercial MPLS 
routers and embedded system routers in terms of hardware architectures and performances which will be 
useful for developing the reconfigurable router testbed and benchmarking its performance. 

The remainder of this paper is structured as follows; Section 2, we discuss on the related work of 
previous router testbed. It is divided into three parts which are software-based, commercial MPLS router, 
and embedded system-based router. Then, in Section 3 and 4, the performance of commercial and software- 
based MPLS router and embedded system-based routers are analyzed respectively. Finally, this paper is 
concluded with Section 5. 


2. RELATED WORK 

Various studies on reconfigurable routers using software simulations, commercial routers and 
embedded system-based routers are reviewed in this section. The routers will be described in detail in terms 
of architecture, structure, and cost effectiveness. The summary of these work will be tabulated at the end of 
this section. 


2.1. Software-based Router Testbed 

This section presents studies that have been conducted to develop a software-based router testbed. 
The main simulators in the literature are Click [16-17], Minos and Data Plane Development Kit (DPDK) 
[18], MATLAB [19] and C++ [20]. 

In 2013, Aldaya et al. [16] created a programmable optical router to test optical switching fabrics by 
using SwitchingMatrix in Click software. Wang et al. [17] further improved Click-based router testbed by 
proposing an additional software which is OpenFlow to their testbed. The testbed achieves high speed 
packets switch forwarding as it utilises personal computers and hubs. Hence, a load-balancing high- 
performance router testbed can be achieved and it can be expanded to support network virtualization. 
This software-based router testbed is also modular, making it easy to conduct network experiment in the 
laboratory for academic purposes. 

In a different work by Xu et al. [18], a framework called Minos has been proposed to regulate router 
actions on data planes. Action Identifier (AID) will be the input for Minos to perform lookups in Regulated 
Action Table (RAT). Finally, Minos will verify whether or not the action is normal. Minos is expected to get 
a couple of distinct factors that will affect the security of a programmable router such as cost and 
effectiveness. This is because the reprogrammability makes the router vulnerable and exposed to the risk of 
being hacked. Minos is implemented and evaluated separately on Click and DPDK. DPDK comprises of a set 
of libraries that support efficient implementations of network functions [21]. 

The worked by Tajiki et al. [19], focuses on a new traffic engineering architecture for SDN-MPLS 
network, where they proposed to improve flow-level management flexibility. It is done by applying 
OpenFlow-enabled switches at the edge of the network while the MPLS routers are the core router of the 
network. The proposed scheme also re-assigns flows in the LSP to highly utilize the network resource. 
The simulation was done by using MATLAB2016b. 

The software router by Goswami et al. [20] studied how to achieve the estimation of optical-layer 
power consumption and cost for a long distance optical networks using Wavelength Division Multiplexing 
(WDM). The test is implemented on two different networks which are IP-over-WDM and IP-over-MPLS- 
over-WDM networks. In the former network test which is IP-over-WDM network, IP routers communicate 
directly with Wavelength-Routed Optical Network (WRON) and bandwidth requests are equal to light-path 
capacity. On the other hand, in IP-over-MPLS-over-WDM network, the IP routers communicate with WRON 
through commercial routers, hence the bandwidth requests arrived as LSP. Therefore, it is lower than light- 
path capacity. The simulation was conducted by using C++ programming language. The advantage of this 
work is that it is cost efficient to run the simulation. However, the results obtained are based on theoretical 
values because there is no hardware involved. 

Despite the fact that it is costs efficient to develop a software-based router, but the parameters in the 
simulations are based on theoretical values. Meaning that, the simulations does not consider noise, non-linear 
effects, and other environmental factors like hardware does. Therefore, hardware emulation is a usual 
progress of testbed development. 


2.2. Commercial Router 

A review of commercial routers will be described in this part. One of the previous work, 
Software Defined Network (SDN) is hybrid with commercial MPLS routers in [22]. The only work that used 
commercial routers alone is by Blair et al. [23]. 
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In a work by Sgambelluri et al. [22], Segment Routing (SR) is implemented in two different 
networks, which are SDN-based and Path Computation Element (PCE)-based. SR technology is proposed to 
enforce an effective routing strategies without relying on signaling protocols. In SDN-based network, 
SR controller is designed to enhance OpenFlow controller. While in PCE-based network, SR is implemented 
where the nodes consist of commercial IP/MPLS routers. This work analysed the performance of both 
proposed implementations. 

Blair et al. [23] used a commercial MPLS router in his study. The authors used four MPLS routers 
in their experiment, with multiple protocols such as IEEE C37.94, IEC 61850-9-2 Sampled Values, and IEC 
61850-8-1 Generic Object-Oriented Substation Event (GOOSE). The study is to demonstrate and analyse the 
use of commercial IP/MPLS protocol to carry protection relay hardware to support power system protection 
functions. They also used Real-time Digital Simulator (RTDS) to simulate the power system and interfaced 
with the hardware protection relays. RTDS allows power system faults and other events to be simulated in 
real-time. 


2.3. Embedded-System based Router Testbed 

The design of embedded system-based router testbeds were mostly designed using Banana Pi [24], 
netFPGA [25] and Raspberry Pi [26]. The main purpose of using embedded system as a router is to create a 
reconfigurable router testbed unlike commercial router. 

Posch et al. [24] proposed a testbed using Banana Pi RI as a router. Banana Pi is a single-board 
computer that has more than one Ethernet ports. This work demonstrates a Named Data Networking (NDN)- 
based multimedia delivery using adaptive bit-rate streaming. Additionally, a graphical user interface is 
provided for the visitors who watch the demonstration to create their own network topology, configure a 
streaming scenario and observe in near-real time. The visitors may also learn which configuration gives the 
best performance of NDN network by selecting different network forwarding strategies and client-based 
adaptation logics. As for the hardware, they used Banana Pi R1, at least two network switches, and a personal 
computer (PC). The testbed required a large space due to large number of components. Banana Pi R1 is using 
A20 ARM Cortex A7 Dual Core as its processor. Then, two dedicated networks which are Management 
Network (MN) and Emulation Network (EN) are connected with Banana Pi R1 in a star topology. The role of 
Management Network (MN) is for configuration and monitoring. While EN is a virtual network overlay that 
is created using networking tools such as iptables and traffic control. The purpose of having the separation 
between MN and EN is to prevent management and control interference with running network emulation. 
However, according to Lech and Wiodarski [27], Banana Pi has its own weakness. It produces a huge delays 
(up to 43 ms) for such a simple point-to-multipoint Local Area Network (LAN) and Wide Area Network 
(WAN) topology. Furthermore, according to the author, Banana Pi can cause a serious problem to the 
network when it receives a large amount of traffic. 

Another study done by Sivarman et al. [25] used netFPGA as their router to study the role of packet 
buffer memory on the power consumption of backbone routers. They have developed an algorithm for the 
memory components to sleep and awake when needed, while being able to control the resulting traffic 
performance degradation in the form of packet loss during congestion. They conducted a comprehensive 
evaluation of their algorithm by using the simulation of offline traffic traces taken from carrier/enterprise 
networks as well as online TCP flow in Network Simulator 2 (NS2). The evaluation also being implemented 
on a programmable router testbed which is the netFPGA. NetFPGA is connected to traffic generators and 
delay emulators to demonstrate the feasibility of implementing the algorithm in the hardware. The advantage 
of using netFPGA is it can be reprogrammable by using VHDL programming language. However, the cost of 
netFPGA is high. 

Gupta et al. [26] has proposed a small size, low cost and portable SDN switch testbed using 
Raspberry Pi. Raspberry Pi is a minicomputer utilizing Advanced Reduced Instrcution Set Computing 
Machine (ARM) as its CPU [28-29]. The testbed reported by Gupta et al. [26] claimed to support OpenFlow 
Specification 1.0 to 1.4. However, the Raspberry Pi only has | Ethernet port but the authors extend it using 
three low-cost USB-based LAN cards. In order to make the Raspberry Pi as SDN switch, it is installed with 
Ubuntu MATE 15.04 instead of Raspbian to support the latest version of OpenFlow switch. Then, four 
laptops are connected to the SDN switch as client, POX/RYU controller, the other two laptops as servers. 
The advantage of using Raspberry Pi as the testbed is it is easy to configure in terms of installing the 
operating system, programming language and the user interface. The price of the Raspberry Pi is also 
reasonable and suitable for beginners. However, the only downside of Raspberry Pi is it only has one 
Ethernet port, therefore it is quite hard to emulate a real network router. 
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3. PERFORMANCE OF COMMERCIAL ROUTER 

The results and performance of the hardware and software-based MPLS routers testbeds will be 
presented in this section. The parameters such as throughput, delay, link utilizations, bandwidth and power 
consumption will be discussed in detail. 

In segment routing (SR) work by [22], SDN-based SR controller uses OFP_FLOW_MOD message 
to configure the nodes while PCE-based SR controller uses PCRep to configure the nodes which are the 
IP/MPLS routers. The scalability performance is tested in two different cases which are single label and 15 
labels. The overall results show that the controller setup time, ingress switch setup time and packet 
forwarding time for 15 labels are greater than single label. This is due to a lot of time required for path and 
label stacking computation. As for PCE-based SR controller, the results are in terms of distribution of PCE 
service time. Comparing with SDN-based SR controller, the two results are similar because the same path 
and segment list computation were implemented in both controllers. However, for both implementation, 
SR controller does not provide the router testbed a rerouting mechanism because it is based on shortest path 
routing. Hence, if the shortest path is congested, the router will always forward the packets on that path. 

Lastly, for work done by [23], the author runs two different experiments. The first experiment is to 
study the delay and jitter of IEEE C37.94 protocol. While the second experiment is to study the comparison 
of protocol trip times and bandwidth between IEEE C37.94 and IEC 61850 SV and GOOSE. For the first 
experiment, the author runs the experiment for nine times. The first five tests are to get the propagation delay 
results for a chain of IP/MPLS routers. According to the results, the jitter buffer size increases as the payload 
size increases. Thus, the propagation delay increases. For sixth test onwards are to get the results of 
propagation delay for ring of IP/MPLS routers. The advantage for the ring setup in sixth test compared to 
fifth test is it has rerouting capability because the setup has an alternate path for the packets in case of broken 
link. There is small difference in terms of propagation delay for the sixth and seventh test due to the variation 
of delay in the hardware. At the eighth and ninth test, the propagation delay increases when number of 
C37.94 slots decreases even when the payload size is very small. This is because the main purpose of the 
slots is to define the end-to-end usable bandwidth for the packets. Hence, if the number of slots is small, 
the usable bandwidth will also be small causing the delay to be high. For the second experiment, the results 
show that IEC 61850 SV and GOOSE are better than IEEE C37.94 in terms of circuit breaker tripped time, 
and backup intertrip time. Even the bandwidth used by these two protocols is greater, they are able to transfer 
much more information for three-phase voltage and current waveforms. Hence, the data acquired is more 
accurate and better for further analysis. 

Therefore, from the performance of the MPLS routers above, we can conclude that MPLS routers 
are better than IP routers because it is scalable [22], protection stability [23], improves bandwidth efficiency 
[23], and low value of delays. 


4. PERFORMANCE OF EMBEDDED SYSTEM-BASED ROUTERS 

This section will present the findings obtained by the embedded system-based router from Section 
2.3, and will be compared in terms of throughput, cost, and processing capability at the end of this section. 

The study carried out by [24] have successfully developed a low-cost router testbed based on 
Banana Pi R1. The testbed can be accessed by Virtual Private Network (VPN), thus the configuration, 
execution and the visualization of the testbed did not require a lot of bandwidth. The multimedia delivery 
scenario can be configured by using the Web interface. In terms of delay, their results presented in [30] show 
that the delay is big despite the packets are sent from one node to its adjecent node via CAT6 Ethernet cable. 
This is because the Banana Pi R1 is connected with two networks, hence the processor cannot keep up with 
the networks, operating systems, and data transmission at the same time. Supported by the results obtained in 
[27], it shows that Banana Pi R1 gives low throughput and high delay for point-to-multipoint connection in 
LAN and WAN network despite it has Gigabit Ethernet ports. To overcome these limitations, suggested 
method to reduce the delay is to reduce the number of nodes because the higher the number of nodes, 
the greater the delay. Furthermore, Banana Pi R1 is suggested to connect to the network via fiber optics 
instead of wireless and Ethernet. This is because fiber optics provide more effective solution for long 
distance data transmission. In terms of topology, the nodes in [24] are connected in star topology. 
The advantage of using this topology is the client can have point-to-point communication with other client 
resulting the transmission delay is lesser than the other topologies. However, this type of topology does not 
have an alternate route for the packets in case of broken links. Hence, an improvement that can be made is to 
use ring topology to prevent data transmission failure in case of broken link as well as link congestion. 

While in [25], the power against loss trade-off performance were evaluated when the algorithm is 
implemented on netFPGA and TCP flows were generated for 180 seconds using iperf. The algorithm has 
saved 40% of the energy when it is under very heavy load. As a comparison, for example, a Raspberry Pi 
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typically can generate around 98Mbps for 10 seconds. Which means, it can generate a huge amount of 
throughput within short period of time compared to netFPGA. This is due to some packet loss as described in 
their work. 

The most recent work in [26] did not present any numerical result. However, they have successfully 
developed and tested the SDN-switch by using load balancing and firewall application. The load balancing 
architecture consists of OpenFlow switch network with POX controller and multiple servers connected to the 
OpenFlow switch. The firewall allows or rejects a specific type of data. In this case, the firewall only allows 
MAC address, IP address, and port number. When the packet arrived at the switch, the firewall check 
whether the packet header matched with its rules or not. The main advantage of this testbed is the hardwares 
are cost friendly since Raspberry Pi were being utilised with a few personal laptops to run the experiment. 
However, this testbed is expected to have a high delay because only one Raspberry Pi is connected to all the 
laptops. This will cause a problem for the Raspberry Pi to keep up all with the data that are transmitted at the 
same time. Furthermore, the topology is also the same as [30] which is star topology. 

From the performance of the said embedded system-based routers, each of the testbeds have their 
own pros and cons affected by the type of traffics, topology and hardware. From the analysis, Raspberry Pi is 
a better choice for a cost-friendly testbed [26], because it can generate high throughput within short period of 
time and it can handle multiple nodes connected to it. 


5. CONCLUSION 

This paper has presented an overview on router testbeds of software-based, commercial, 
and embedded system-based. These testbeds have similar method of conducting the experiment which are 
using hardwares, softwares, or both. For the software-based testbeds, the simulations achieved customizable, 
programmable, and has the capability to emulate a complex routing network while achieving a high- 
performance testbed. However, the results obtained are based on ideal case and not involving non-linear 
effects. As for commercial routers, the results show that it is better than traditional IP-based routers because 
it can provide better throughput, protection, bandwidth and link efficiency, and delays despite it consumed 
higher power. However, commercial routers do not have the freedom to reconfigure for academic purposes. 
Last but not least, design of a completely reconfigurable and reprogrammable router testbed can be achieved 
by using embedded system hardware making it the most suitable choice for academic purposes. However, 
they have limited processing unit capability causing the performance of the network is lesser than using 
industrial routers. Therefore, our next study is to develop a reconfigurable MPLS router testbed by using 
embedded system-based hardware, which will be utilising analysis from this paper as the basis. The MPLS 
router will be useful for architecture and performance benchmarking. Hence, the plan for the future work is 
to validate the performance results of the developed reconfigurable router with the industrial grade MPLS- 
based router. 
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